Image Semantic Segmentation Network Based on Semantic Propagation and Fore-Background Aware
LIU Zhanghui1,2, ZHAN Xiaolu1,2, CHEN Yuzhong1,2
1. College of Computer and Data Science, Fuzhou University, Fuzhou 350116; 2. Fujian Provincial Key Laboratory of Networking Computing and Intelligent Information Processing, Fuzhou University, Fuzhou 350116
Abstract：Although image segmentation is widely applied in many fields owing to the assistance of better analysis and understanding of images, the models based on fully convolutional neural networks still engender the problems of resolution reconstruction and contextual information usage in semantic segmentation. Aiming at the problems, a semantic propagation and fore-background aware network for image semantic segmentation is proposed. A joint semantic propagation up-sampling module(JSPU) is proposed to obtain semantic weights by extracting the global and local semantic information from high-level features. Then the semantic information is propagated from high-level features to low-level features for alleviating the semantic gap between them. The resolution reconstruction is achieved through a hierarchical up-sampling structure. In addition, a pyramid fore-background aware module is proposed to extract foreground and background features of different scales through two parallel branches. Multi-scale fore-background aware features are captured by establishing the dependency relationships between the foreground and background features, thereby the contextual representation of foreground features is enhanced. Experiments on semantic segmentation benchmark datasets show that SPAFBA is superior in performance.
[1] LONG J, SHELHAMER E, DARRELL T.Fully Convolutional Networks for Semantic Segmentation // Proc of the IEEE Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2015: 3431-3440. [2] 李艳丽,周忠,吴威.一种双层条件随机场的场景解析方法.计算机学报, 2013, 36(9): 1898-1907. (LI Y L, ZHOU Z, WU W.Scene Parsing Based on a Two-Level Conditional Random Field. Chinese Journal of Computers, 2013, 36(9): 1898-1907.) [3] 赵雪梅,李玉,赵泉华.基于隐马尔可夫高斯随机场模型的模糊聚类高分辨率遥感影像分割算法.电子学报, 2016, 44(3): 679-686. (ZHAO X M, LI Y, ZHAO Q H.Hidden Markov Gaussian Random Field Based Fuzzy Clustering Algorithm for High-Resolution Remote Sensing Image Segmentation. Acta Electronica Sinica, 2016, 44(3): 679-686.) [4] ZHAO H S, SHI J P, QI X J, et al. Pyramid Scene Parsing Network // Proc of the IEEE Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2017: 6230-6239. [5] CHEN L C, PAPANDREOU G, SCHROFF F, et al. Rethinking Atrous Convolution for Semantic Image Segmentation[C/OL].[2021-07-30]. https://arxiv.org/pdf/1706.05587.pdf. [6] YANG M, YU K, ZHANG C, et al. DenseASPP for Semantic Segmentation in Street Scenes // Proc of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2018: 3684-3692. [7] HUANG G, LIU Z, VAN DER MAATEN L, et al. Densely Connected Convolutional Networks // Proc of the IEEE Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2017: 2261-2269. [8] HU J, SHEN L, ALBANIE S, et al. Squeeze-and-Excitation Networks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020, 42(8): 2011-2023. [9] ZHANG H, DANA K, SHI J P, et al. Context Encoding for Semantic Segmentation // Proc of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2018: 7151-7160. [10] WANG X L, GIRSHICK R, GUPTA A, et al. Non-local Neural Networks // Proc of the IEEE Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2018: 7794-7803. [11] FU J, LIU J, TIAN H J, et al. Dual Attention Network for Scene Segmentation // Proc of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2019: 3141-3149. [12] HUANG Z L, WANG X G, HUANG L C, et al. CCNet: Criss-Cross Attention for Semantic Segmentation // Proc of the IEEE/CVF International Conference on Computer Vision. Washington, USA: IEEE, 2019: 603-612. [13] LI X, ZHONG Z S, WU J L, et al. Expectation-Maximization Attention Networks for Semantic Segmentation // Proc of the IEEE/CVF International Conference on Computer Vision. Washington, USA: IEEE, 2019: 9166-9175. [14] ZHU Z, XU M D, BAI S, et al. Asymmetric Non-local Neural Networks for Semantic Segmentation // Proc of the IEEE/CVF International Conference on Computer Vision. Washington, USA: IEEE, 2019: 593-602. [15] ZHANG F, CHEN Y Q, LI Z H, et al. ACFNet: Attentional Class Feature Network for Semantic Segmentation // Proc of the IEEE/CVF International Conference on Computer Vision. Washington, USA: IEEE, 2019: 6797-6806. [16] YUAN Y H, CHEN X L, WANG J D.Object-Contextual Representations for Semantic Segmentation // Proc of the European Conference on Computer Vision. Berlin, Germany: Springer, 2020: 173-190. [17] CHEN Y P, ROHRBACH M, YAN Z C, et al. Graph-Based Global Reasoning Networks // Proc of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2019: 433-442. [18] WU T Y, LU Y, ZHU Y, et al. GINet: Graph Interaction Network for Scene Parsing // Proc of the European Conference on Computer Vision. Berlin, Germany: Springer, 2020: 34-51. [19] 张焯林,赵建伟,曹飞龙.构建带空洞卷积的深度神经网络重建高分辨率图像.模式识别与人工智能, 2019, 32(3): 259-267. (ZHANG Z L, ZHAO J W, CAO F L.Building Deep Neural Networks with Dilated Convolutions to Reconstruct High-Resolution Image. Pattern Recognition and Artificial Intelligence, 2019, 32(3): 259-267.) [20] RONNEBERGER O, FISCHER P, BROX T.U-Net: Convolutional Networks for Biomedical Image Segmentation // Proc of the International Conference on Medical Image Computing and Computer-Assisted Intervention. Berlin, Germany: Springer, 2015: 234-241. [21] NOH H, HONG S, HAN B.Learning Deconvolution Network for Semantic Segmentation // Proc of the IEEE International Confer-ence on Computer Vision. Washington, USA: IEEE, 2015: 1520-1528. [22] WU H K, ZHANG J G, HUANG K Q, et al. FastFCN: Rethinking Dilated Convolution in the Backbone for Semantic Segmentation[C/OL].[2021-07-30]. https://arxiv.org/pdf/1903.11816v1.pdf. [23] TIAN Z, HE T, SHEN C H, et al. Decoders Matter for Semantic Segmentation: Data-Dependent Decoding Enables Flexible Feature Aggregation // Proc of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2019: 3121-3130. [24] ZHANG H, ZHANG H, WANG C G, et al. Co-occurrent Features in Semantic Segmentation // Proc of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2019: 548-557. [25] HE J J, DENG Z Y, QIAO Y.Dynamic Multi-scale Filters for Semantic Segmentation // Proc of the IEEE/CVF International Conference on Computer Vision. Washington, USA: IEEE, 2019: 3561-3571. [26] ZHONG Z L, LIN Z Q, BIDART R, et al. Squeeze-and-Attention Networks for Semantic Segmentation // Proc of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2020: 13062-13071. [27] HE J J, DENG Z Y, ZHOU L, et al. Adaptive Pyramid Context Network for Semantic Segmentation // Proc of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2019: 7511-7520. [28] HU H Z, JI D Y, GAN W H, et al. Class-Wise Dynamic Graph Convolution for Semantic Segmentation // LAI Y X, WANG T, JIANG M, et al., eds. Lecture Notes in Computer Science. Berlin, Germany: Springer, 2020, 12362: 1-17. [29] CHEN W L, ZHU X G, SUN R Q, et al. Tensor Low-Rank Reconstruction for Semantic Segmentation // Proc of the European Conference on Computer Vision. Berlin, Germany: Springer, 2020: 52-69. [30] LIU J B, HE J J, ZHANG J W, et al. EfficientFCN: Holistically-Guided Decoding for Semantic Segmentation // LAI Y X, WANG T, JIANG M, et al., eds. Lecture Notes in Computer Science. Berlin, Germany: Springer, 2020, 12371: 1-17.
2022, Vol. 35 
